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This report presents a first year evaluation of the 
Computer Assisted Instruction (CAI)/ Cooperative Learning Project, a 
3-year collaborative effort by two Pennsylvania school districts — the 
Pittston Area School District and the Hatboro-Horsham School 
District — and Research for Better Schools (RBS) . The project, proposed 
to integrate advanced integrated learning system technology with 
cooperative teaching in the classroom and collaborative learning in 
the home* The integrated learning system included inquiry-based, 
hypermedia learning and cooperative learning techniques. It was 
anticipated that the project could potentially serve as a nodel for 
effective CAI that could be nationally validated, disseminated, and 
adopted by school districts across the nation. The project was to 
include the acquisition of computer hardware and software, initial 
and on-going teacher training, achievement and attitudinal data 
collection and analysis,- and dissemination. The evaluation of the 
study was designed to determine the extent to which the program: (1) 
enhances mathematics and language arts achievement for the students; 

(2) produces an increase in time-on-task behavior for the students; 

(3) enhances student and parent attitudes toward learning, the 
school, and education in general; and (4) enhances positive teacher 
and administrator attitudes toward the integrated learning system and 
cooperative learning. Data for the first year evaluation were 
gathered via administrator, teacher, and parent questionnaires; 
however, a pretest-posttest control group design will be used for 
years 2 and 3 to enhance the validity of the findings. The 
introductory section of the report provides background information on 
the project and the evaluation questions addressed by the study. 
Later sections describe the evaluation design and procedures, 
preliminary findings, and some conclusions and implications based on 
these findings. Appended copies of the questionnaires for each of the 
three groups include tallies of the responses. (DB) 
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Int roduction 

This report represents a first-year evaluation of the "CAI/Cooperative 
Learning Project." The three-year project is a collaborative effort by two 
Pennsylvania school districts, the Pittston Area School District and the 
Hatboro-Horsham School District, and Research fos. .tter Schools (RBS). It 
is federally supported by an Innovation in Education Program Grant. 

The introductory section of the report provides some background 
information on the project and the evaluation questions addressed by the 
study; later sections of the report describe the evaluation design and 
procedures, preliminary findings, and some conclusions and implications 
based on these findings. 
Background 

Both the Pittston and Hatboro-Horsham School Districts had previously 
been involved in successful efforts to implement computer-assisted 
instructional technology. In order to further adopt this technology, in 
combination with innovative educational practices, the two districts jointly 
proposed this collaborative project, under the Fund for Innovation in 
Education. Specifically, the project proposed to integrate advanced 
integrated learning system (IL.S) technology with cooperative teaching in the 
classroom and collaborative learning in the home. It was anticipated that 
the project could potentially serve as a model for effective 
computer-assisted instruction that could be nationally validated, 
disseminated, and adopted by 9chcol districts across the nation. To realize 
this potential, the project needed a sound evaluation plan capable of 
demonstrating the full extent of its effectiveness. Because of its history 
in evaluating computer-assisted instructional programs, RBS was invited to 
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collaborate with the two districts in the project, serving as a third-party 
evaluator . 

The project was to include the acquisition of computer hardware and 
software, initial and on-going teacher training, achievement and attitudinal 
data collection and analysis, and dissemination* For their software, the 
two school districts selected a computer-based learning system marketed by 
Jostens Learning Corporation. 
Evaluation Questions 

The purpose of the CAI/Cooperative Learning Project is to develop a 

replicable, independently validated, national model for integrating 

computer-assisted instruction through an integrated learning system that 

includes inquiry-based, hyper-media learning and cooperative learning 

techniques. An original feature of the project is the inclusion of 

cooperative education, creating a collaborative learning environment both 

within the classroom and at home. Based on the project goals, as specified 

in the project proposal, four outcome evaluation questions were formed to 

focus the evaluation study. They were: 

1* To what extent does the program enhance mathematics and language 
arts achievement for the participating students? 

• To what extent is there an increase in the number of students in 
each school meeting and/or exceeding state standards on the TELLS 
test? 

• To what extent is there an increase in scores on the Stanford 
Achievement Tedt (SAT) for participating Pittston students and on 
the Iowa Test of Basic Skills (ITBS) for participating 
Hatboro-Horsham students? 

2. To what extent does the program prouuce an increase in time-on-task 
behavior for participating students? 

3. To what extent does the program enhance student and parent attitudes 
toward learning, the school, and education in general? 
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4. To what extent does the program enhance positive teacher and 

administrator attitudes toward the integrated learning system and 
cooperative learning? 

In early discussions with the two project directors, it was decided 
that the evaluation for the project's first year should be primarily 
descriptive and focus on process and outcome concerns. An implementation 
focus for the first year study would help to insure that prerequisite 
conditions for proper program implementation were met. This is particularly 
important because, due to delayed funding and difficulties in ordering and 
receiving the necessary hardware and software, the project was not initiated 
until the second half of the school year. In addition, complex programs 
such as these, which require considerable equipment, training, and effort 
are frequently not fully operational in their first year. And even among 
programs which are operational, it is unrealistic to expect significant 
impact after only one year of implementation. However, it should be noted 
that a limitation of this first year study was having to rely primarily on 
surveys as implementation indicators. Also, cooperative learning was not 
included as part of the process evaluation as the project directors decided 
that it was not feasible to implement this component during year one. 

The outcome component of the evaluation was designed to insure that the 
program and control groups at each site were comparable. This would help to 
rule out some plausible rival hypotheses for pretest-posttest differences 
when additional outcome data are collected in years two and three. Thus, 
the process and outcome evaluation questions which formed the focus of this 
first year study were as follows: 

• To what extent was the project implemented as planned? 

• To what extent were comparable control and experimental groups 
established within each of the districts and baseline data 
collected? 
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Evaluation Design and Procedures 
This section describes the evaluation resign, student sample, 
instrumentation, and data collection and analysis undertaken by RBS in 
connection with its first year evaluation of the CAI/Cooperative Learning 
Project implemented in two middle schools, oae in the Pitts ton Area School 
District and one in the Hatboro-Horsham School District. 
Design 

The approach used to address the evaluation questions was to include 
quantitative data collection of curriculum-embedded tests, standardized 
achievement tests, and the TELLS. In addition, implementation and attitude 
measures were to be obtained through survey data collected at the beginning 
and end of the school year and through observations and informal interviews 
made during periodic visits to each of the sites. A pretes t~posttest 
control group design was to be used (in years two and three) to enhance the 
validity of the findings, although there were differences in the nature of 
the control or comparison group at each school. For the Hatboro-Horsham 
School District, a control group was to be constituted from students within 
the same school who had no contact with computer-assisted instruction , For 
the Pittston Area School District, a control group was to consist of 
students within the school who had minimal contact with computer-assisted 
instruction. 
Student Sample 

The selection of the program and control group differed at each site. 
In the Hatboro-Horsham middle school each grade is divided into two 
heterogeneous teams, a red team and a black team. The program group was 
selected from only the red team and consisted of the 25 lowest-achieving 
students from each of the sixth, seventh, and eighth grades. All of these 
students had standardized achievement scores below grade level in 



mathematics. The control group consisted of 25 comparable students from 
each of the black team's three grades (these students did not have computers 
in their classes) ♦ 

The Pittston middle school was divided into two groups based on 
standardized achievjment test scores, an at-risk or program group (i.e., 
scoring below the grade level median) and a not at-risk group. All students 
in the at-risk group were then randomly assigned to one of two groups, a 
"low use" group which received 15 minutes of computer time per we* c, or a 
"high use" group which received 60 minutes of computer time per week (in 15 
minute segments). There were approximately 50 students in each group at 
each grade level. To balance the low and high use groups, students assigned 
to one group for mathematics v \11 h? assigned to the other group for 
reading. It should be noted that although the not at-risk group was also 
divided into low and high use groups, their data are not included in this 
report as the initial focus of the project was to be on low-achieving 
students. In both sites, this first year implementation was only for 
mathematics* Also, in each school program students (five at a time) were on 
a rotating schedule to bring a computer home for a six week period. 
Instrumentation 

A number of evaluation instruments were developed by RBS, and approved 
by the districts, for use in the three-year study. They included the 
following: 

• Administrator Attitude Survey (pre and post) 

• Teacher Attitude Survey (pre and post) 

• Student Attitude Survey (pre and post) 

• Parent Attitude Survey (pre and post) 

• Computer Home Use Survey (pre and post) 

• RBS Time-on-Task Measure. 

The administrator and teacher attitude forms contain questions which 
address concerns regarding the new computer learning system, the advantages 
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and disadvantages of the system, the adequacy of training and follow-up 
technical assistance, the implementation of the system, the adequacy of the 
computer curriculum, and the effectiveness of the computer learning system. 
The student attitude form addresses attitude toward using the computer in 
school, and the advantages and disadvantages of the computer learning 
system; and the parent attitude form and the home use form ask about parent 
involvement in the program, their child's attitude toward participation, and 
the advantages and disadvantages of the computer learning system* The home 
use survey is developed for parents of those students, who on a rotating 
basis take a computer home to increase their time on the system and their 
parents' involvement in collaborative learning. The time-on-task measure 
was developed and validated by RBS to structure observations for collecting 
data on student engaged time. 
Data Collection and Analysis 

Both process and outcome (i.e., baseline) data were collected from the 
two sites. Process data were collected through attitude surveys, and 
informal interviews and observations. Because of the delays in start-up 
previously mentioned, most surveys were administered only once during the 
school year (pre-post data colle^* on is planned for years two and three) 
and time-on-task measures were not obtained. Twice after the program was 
initiated, RBS met with the project director and computer coordinator from 
each district. On the second occasion, visits were made to a number of 
classrooms in each school to observe students working on the computer and 
interviews were conducted with the computer coordinator and teachers. 

The baseline data submitted by the two districts consisted of 
standardized achievement test scores (the Stanford Achievement Test (SAT), 
for Pittston, and the Iowa Test of Basic Skills (ITBS), for 



Hatboro-Horsham) , and Basic: Skills Inventory (BSI) data which reflected 
students* initial placement in the computer learning system's curriculum. 
As noted earlier, this data will be submitted at the beginning and end of 
years two and three to analyze achievement gains* The TELLS data were not 
submitted to RBS, as originally proposed, as the districts were waiting for 
the state's dec. f 3 ion as to whether or not this testing program would 
continue. In the proposal, separate analyses were planned for process and 
outcome data from .raUi district. However, since each site was only in 
partial operation for part of the 1990-91 school year, this report includes 
process and baseline data; posttest data were not collected and analyzed. 
As previously indicated, it was agreed to be premature to look at student 
achievement growth. The revised analysis plan for this first year was to 
analyze survey and interview data related to program implementation to 
provide information for program improvement planning for years two and 
three, and second, to examine baseline data to see che extent to which 
control and program groups were comparable. 

Findings 

The findings presented in this section of the report relate to the 
revised evaluation questions for year one. They are discussed in two 
separate sections, one for each evaluation question. 
To What Extent was the Project Implemented as Planned 7 

In the fall of the 1990-91 school year, after some delay, each of the 
two sites ordered and received the computer hardware and software necessary 
for implementing the project; teachers were then provided with training and 
technical assistance for implementing the Jostens mathematics curriculum; 
program and control groups were identified; and schedules were developed for 
data collection and student computer use. During the second half of the 
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school year, baseline achievement and attitudiral data were collected 
and analyzed and identified students began to implement the mathematics 
portion of the Jos tens curriculum. In adclitior , parental involvement was 
initiated at the two middle schools through providing information on the 
innovative program, requesting permission for their childs* participation, 
and through the initiation of the home computer use component. 

Comprehensive planning was also underway for year two of the project, 
the 1991-92 school year. This included developing schedules for training 
teachers on the Jostens reading curriculum, developing and revising 
schedules for student computer use, and selecting program and control 
students for the new sixth grade students entering the program. Also, in 
discussions with RBS, plans were developed for periodic site visits and for 
collecting the data necessary to address the study's evaluation questions. 
Although the project's cooperative learning component had not been 
implemented, each site began exploring ways to phase in this component in 
coordination with a larger district effort. 

Attitude Data , Attitude surveys were administered to administrators, 
teachers, students, and parents during the second half of the school year. 
These data are briefly described below, and presented separately for each 
district. A summary of all quantifiable survey data (i.e., mean ratings) is 
presented in Appendix A. Although some surveys were administered on more 
than one occasion, change data are not discussed here, as tb* average time 
span between these measures was only about three months. 

Two administrators in the Hatboro-Horsham district completed the 
survey. Both agreed to having some experience in ^sing computers and felt 
adequately trained for their role in the use of the computer system. They 
also agreed that the training adequately prepared the;- statx for use of the 
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computer system and that the follow-up technical assistance provided after 
training met staff needs. Of the 20 potential concerns about the new 
computer learning system listed on the survey form, one administrator did 
not agree with any and the other administrator indicated concern with only 
four of the items listed (i.e., a rating of 4 or 5 on the 5-point scale): 
the changes this system will cause teachers to make in their classrooms, the 
scheduling of students on the system, the follow-up technical assistance on 
the computer learning system, and the alignment of the system's curt Lculum 
and the district's curriculum and testing. In their comments, the 
administrators indicated major advantages of the computer learning sj'stem to 
be: provides individualized instruction, enables teachers to bs mor^ 
effective, and provides data for instructional decisions. Major problems 
listed were: limited hardware and need for additional teacher training. 

The two Pittston administrators who completed the survey reported 
having no experience in using computers and did not participate in any 
training on using the new computer system. One administrator felt that the 
staff training adequately prepared them for using the computer system and 
the other administrator reported that the training adequately covered 
mathematics, but not reading. In contrast to the Hav.boro-Horsham 
administrators ♦ these respondents agreed with the majority of the 20 
potential concerns listed on the survey (i.e., a rating of 4 or 5 on the 
5-point scale). The items riot of concern were: student attitude toward the 
system, knowledge that other approaches work better, their personal limited 
knowledge of computers, the effects of the system on students, and the value 
of instruction students receive on the system. Based on their understanding 
of the computer learning system, the aiministrators listed its major 
advantages to be the opportunity to help slow learners, the positive impact 



on student motivation and interest in school work, and students* increased 
access to materials . They reported no major disadvantages to the system and 
the only recommendation was to "speed up the timetable." 

Of the nine teachers in the Hatboro-Horsham middle school who were 
administered the survey, six had some prior computer experience, one had 
very little experience, and two reported having more extensive computer 
experience (mean rating of 3.1 on a 5-point scale). In terms of their 
preparation for using the new computer system and for covering the 
curriculum implemented (i.e., mathematics), aix teachers felt somewhat 
prepared, one teacher felt well prepared, and two teachers indicated that 
they were not adequately trained. Three of the 20 concerns about 'he new 
computer learning system listed on the survey were felt by the majority of 
respondents to be strong concerns (i.e., a rating of U or 5 on the 5-point 
scale). They were: a concern about how much time and paperwork the computer 
system takes, a concern about the scheduling oa. students, and a concern 
about how well the computer system's curriculum aligns with the district 
curriculum and testing. The teachers listed the major advantages of the 
computer learning system to be: individualization, reinforcement, 
motivation, and the fact that it's a new approach. In their comments, they 
listed the major disadvantages of the system to be: scheduling, the loss of 
class time for students working on computers, and the limited number of 
computers . 

Six teachers in the Pitts ton middle school completed the survey; five 
reported having some computer experience and one teacher reported having 
little computer experience (mean rating of 2.8 on a 5-point scale). These 
teachers did not respond to survey items concerning the training and 
technical assistance received in preparation for using the system (perhaps 
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the staff training was still being conducted at the time of the survey). In 

terms of the 20 concerns listed on the survey, no concern received strong 

agreement from more than one or two teachers (i.e., a rating of 4 or 5 on 

the 5-point scale or a mean of 3.0 or higher). Interestingly, the Pittston 

teachers listed major advantages (individualization, alternative approach) 

and disadvantages (scheduling, loss of class time) of the new computer 

learning system similar to those listed by the Hatboro-Horsham teachers. 

The student survey was completed by 68 students in the Hatboro-Horsham 

middle school and 150 students (66 from the Low Use group and 84 from the 

High Use group) in the Pittston middle school. Overall, students in both 

« 

districts gave very positive responses to the survey's 30 "yes-no M items. 
They indicated that they liked using the computer at school, that the 
computer was easy to use, that computers make it fun to learn, and that they 
can do most of the computer lessons without help from anyone else (05 
percent or more of the itudents in each district responded positively to 
these items). Seventy-five percent of the Hatboro-Horsham students and 
sixty-five percent of the Pittston students indicated that the computer 
helped them to learn math better, and over ninety percent of the students in 
each district indicated that they liked computer work better than written 
assignments. More than half of the students in each group reported that the 
computer lessons help them to do work in the classroom better. When asked 
if their parents think they are learning from the computer, 53 percent of 
the Hatboro-Horsham students and 63 percent of the Pittston students 
responded positively. 

The involvement of parents was felt to be an important component of the 
project. Parents learned about the project through an initial letter 
describing the project and requesting permission for their child's 
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participation and through progress reports included in parent conferences 
and, in the Pittston district, attached to student report cards. In 
addition, parents were to receive r general survey and a home use survey. 
However, the results for the parent survey are not included in this report 
as both districts felt it was premature to look at parent responses; the 
Pittston district did not administer the survey and, although it was 
administered in the Hatboro-Horsham district, many parents commented that 
they were uncomfortable responding at this time. Nine parents of children 
who took computers home to use as part of the project (five in 
Hatboro-Horsham and four in Pittston) completed and returned the home use 
survey. Overall, these parents* indie ted that their children frequently 
worked on the computer and seemed to enjoy and benefit from their computer 
work. Most parents reported that they did not work with their child on the 
computer system. 

To What Extent Were Comparable Control and Experimental Groups Established 
Within Each of the Districts and Baseline Data Collected ? 

In interviews with the computer coordinator and project director in 
each district, they indicated that the control groups were formed by a 
combination of matching and randomization. Pretest or baseline data for the 
program and control students were submitted to RBS and summarized. These 
data are summarized in the following paragraphs. 

Achievement data . The first type of achievement data, presented in 
Table 1, is embedded testing data from the computer system curriculum. It 
should be noted that only the mathematics curriculum way implemented, thus 
reading placement data are not included. The rationale for looking at this 
data is that, if the computer learning system is to have an impact on 
student achievement in general, it must first be able to demonstrate a more 
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Table 1 



Student Basic Skills Inventory (BSI) Scores by Grade 



School/Grade 



N 



Group 



Mean Placement 
(Grade Level) 



HATBORO-HORSHAM MS 
6 
7 



25 
16 
18 



Program 
Program 
Program 



4.0 
3.8 
4.5 



PITTSTON MS 



49 
45 
47 
51 
34 
54 



Low Use 
High Use 
Low Use 
High Use 
Low Use 
High Use 



4.1 
3.9 
4.5 
4.2 
4.9 
4.9 



Note : BSI scores are for mathematics only; implementation of the reading 
curriculum is planned for year two. The two Pittston Groups (Low Use and 
High Use) represent two levels of program participation (i.e., 15 
minutes/week of computer time or 60 minutes/week computer time). 
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immediate impact on the embedded testing ih the curriculum. .s the table 
indicates, the mean placenent level on the Basic Skills Inventory (the CAI 
embedded test) for each grade was at a level below students' actual grade 
level. In addition, both computer coordinators commented that some initial 
placements had to be upwardly adjusted in order to ensure that students were 
being challenged at appropriate levels. 

Standardized achievement test scores will be collected, on a pre-post 
basis, in order to see if progress on the computer system curriculum is 
transferable to standardized achievement tests. Table 2 presents this 
baseline data (SAT and IOWA) for program and control groups in the two 
districts. Listed in the table are mean NCE scores, standard deviations 
(SD), and a t test for independent sample, for mathematics and reading at 
each grade level. NCE comparisons indicate how well children in these two 
schools fare achievement-wise in comparison to the rest of the country. As 
the table indicates, prior to implementation of the program most students 
(program and control) were performing at below grade level or below average 
(an NCE of 50 is at grade level). The results of the t tests indicate that 
the program and control groups were similar with respect to achievement 
scores. And, in the 2 cases where there were pretest differences (reading 
scores for Hatboro-Horsham grade 8 and Pittston grade 6), analysis of 
covariance can be used to make adjustments in analyzing achievement gains. 

Conclusions and Implications 

Although delays in the acquisition of hardware and software for the 
computer learning system prevented the two schools from starting to 
implement the program on schedule, by the end of the school year the 
mathematics portion of the program was implemented, as planned. Baseline 
data was collected and inclicat d program and control groups were properly 



Table 2 



Student Achievement Scores for Program and Control Groups 

(NCE Scores) 



School/Grade 


N 


Group 


Mathematics 
Mean SD t 


Reading 
Mean SD t 


HATBORO-HORSHAM MS 
















6 


23 


Program 




12 


ns 


46 


11 ns 




22 


Control 


46 


11 




42 


14 


7 


20 


Program 


38 


8 


ns 


39 


12 ns 




25 


Control 


38 


8 




44 


9 


8 


17 


Program 


37 


14 


ns 


40 


10 -2.12* 




20 


Control 


40 


13 




47 


12 


PITTSTON MS 
















6 


49 


Low Use 


50 


14 


ns 


41 


10 -2.05* 




45 


High Use 


51 


14 




46 


13 


7 


45 


Low Use 


46 


9 


ns 


43 


11 ns 




51 


High Use 


46 


10 




46 


10 


8 


33 


Low Use 


49 


12 


ns 


48 


14 ns 




56 


High Use 


45 


15 




46 


15 



Note ? Hatboro-Horsham scores are for the Iowa Test of Basic Skills (ITBS); 
Pittston scores are for the Stanford Achievement Test (SAT). The two 
Pittston groups (Low Use and High Use) represent two levels of program 
participation (i»e. t 15 minutes/week of computer time or 60 minutes/week 
computer time) . 
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selected and comparable. For years two and three, additional adjustment for 



pretest differences which might confound differences between the program and 
control groups will be made statistically (e.g., using analysis of 
covariance). However, for this first year of the project, the shortened 
implementation made difficult the analysis of any impact; that is, to expect 

idence of meaningful student achievement gains in less than a year (and 
some say, not mitil a program has stabilized). 

This first evaluation has several implications for the project in years 
two and three. 

• Implementation of ^omputer-assisted instructional programs commonly 
experience delays the acquisition of resources which are beyond 
the control of loca,. districts. To the extent possible, these 
delays should be anticipated, particularly in planning with vendors. 

• A program of this complexity should not expect all components to be 
implemented concurrently (e.g., different curriculum, home 
component, cooperative learning)- Perhaps planning for a number of 
implementation phases or steps would be more realistic and would 
result in fewer concerns about change for those involved. 

• Program staff must continue to assess and be responsive to teacher 
(administrator, and parent) concerns, particularly with regard to 
training and scheduling. However, RBS has found in evaluating 
similar computer-assisted instructional programs, that teachers* 
initial concerns tend to result in much more positive attitudes over 
time . 

• Student placement in the computer learning system should be closely 
monitored as it was reported by the districts that some students 
were placed far below grade level. This below grade level placement 
also has implications for any expected impact on standardized test 
scores, i.e., to expect an impact, the subject matter for the 
curriculum and test must be related. 

• The number of students in the comparable program and control groups 
at each school should be the same, or as close as possible, and 
efforts should be made to obtain data from all identified students. 
Although statistical procedures can control for small group 
differences, the eighth grade at the Pittston middle school had 
almost twice as many students in the High Use group (56) as in the 
Low Use group (34) . 

• Use of the pretest-posttest control group design in years two and 
three controls for most threats to internal validity and thus 
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eliminates alternative hypotheses for changes in student 
achievement. However, the one threat not controlled for, and which 
should be closely monitored by the two sites, is whether there is 
any change in program and control group composition in the two 
middle schools . 

The cooperative learning component of the program needs to be 
adequately addressed* That is, either plans must be developed for 
its implementation and evaluation or the project (e.g., goals, 
title) should be revised to reflect the elimination of this major 
component from the design. 

The parent involvement component of the program needs to be informed 
and guided by each child's teacher in order to optimize student 
impact. In addition to general information on the project, parents 
need information on how to interpret the computer learning system 
reports and the project's expectations for parental participation 
and support. This is particularly critical if a home application 
component is to be introduced and evaluated. 
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APPENDIX 

Mean Survey Responses for Rated Items 
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Administrator Survey 
Hatboro-Horsham School District (W=2) 



A. 4. 

B. l. 

B.2. 

B.3. 
B.4. 
B.5. 

B.6. 

B.7. 

B.8. 

B.9. 

B.10. 

B.ll. 
B.12. 

B.13. 

B.14. 

B.15. 

B.16. 

9 
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Item 

To what extent are you experienced in using 
computers? 

I don't know enough about the computer learning 
system, 

I am concerned about the attitude of students toward 
the computer system 

I know of other approaches that might work better. 

I have very limited knowledge about computers. 

I am concerned about the effects of the system on 
students . 

I am concerned about the changes this system will 
cause teachers to make in their classrooms. 

I am concerned about how much time and paperwork 
the computer system takes. 

I am concerned about how the scheduling of students 
on the system works out. 

I am concerned about the .ime and energy commitments 
the computer system requires. 

I am concerned about the knowing how to use the 
hardware . 

I am concerned about knowing how to use the software. 

I am conceri ed about the curriculum content of the 
computer learning system. 

I am concerned about the instructional approach used 
by the computer learning system. 

I am concerned about the training I receive on the 
computer learning system. 

I am concerned about the follow-up technical 
assistance I receive on the computer learning system. 

1 am concerned about hnw well the computer system's 
curriculum aligns with our district curriculum and 
testing. 




B . 17 . I am concerned about the use of computer printouts 2.5 
from the system. 



B.18. I would like to know how this system is better than 2.0 
what we had before. 

B.19. I am not interested in learning more about the system, 2,0 

3,20. I am concerned about the value of instruction 2,0 
students receive on the system. 

D.2. To what extent do you feel the training adequately 3,5 
prepared you personally, for your role in use of the 
computer system? 

D.4. To what extent do you feel the training adequately 4.0 
prepared your staff for use of the computer system? 

D.5. To what extent has the inservice training your staff 
received adequately covered the following areas? 

Reading 3,5 

Math 4 - 5 

Writing 3,0 

Science 3,0 

Higher Order Thinking Skills 2,5 

Use of Student results reports 3.0 

Integration of computer curriculum with classroom 2.5 
program 

Hands-on experience with the computer system 4.0 

Scheduling of students 3,5 

D,7, To what extent do you feel the follow-up technical 4,0 
assistance provided after training was adequate to 
meet the needs of you and your staff? 

D.8. To what extent has there been a need for follow-up 
technical assistance in the following areas? 

Reading 3 . 5 

Math 4-0 1 

Writing 5.0 1 

Science 4.0 

Higher Order Thinking Skills 2.5 

Use of Student results reports 3.0 

Integration of computer curriculum with classroom 2.5 
program 

Hands-on experience with the computer system i .0 

Scheduling of students 3.0 
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D.9. 



To what extent have the following documents met your 
staff needs? 



Curriculum Guioes 4.0 
Teacher Guides 4.5 
Teacher Handbooks 4.5 

E.2. To what extent are the reports clearly organized anu 
easy to use? 

E.3* To what extent have the reports enabled your staff to: 

Diagnose student needs 4.0 

Individualize instruction 4.0 

Group students by ability 3.0 

Inform parents of student progress 3.0 

E.4. To what extent have the reports been useful to you 3.0 
as an administrator? 

E.5. To what extent has the computer system resulted in 
reallocation of time in the following areas? 

Reading 3.0 

Math 3.0 

Writing 2.0 1 

Science ~~ 2 

E.6. To what extent has the computer system resulted in 
increased time~on~task for all students in the 
following subject areas? 

Reading 4.5 
Math 4.5 
Writing 4.0 1 

Science ~~ 2 

E.7. To what extent do students enjoy working on their 4.5 
computer lessons? 

E.8. To what extent do you observe students working on 4.5 
lessons at the computer? 

E.9. To what extent has the computer system hardware been 4.0 
reliable and dependable? 

E.10. To what extent has implementation and operation of 1.5 
the computer system created administrative problems 
for you? 
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f.i 



To what extent is the computer curriculum sufficiently 
comprehensive to provide adequately for the learning 
needs of students in eac ! \ of the following subjects? 



Reading 4.0 1 

Math 4.0 1 

Writing 4.0 1 

Science 2 

F.2. To what extent are the learning strategies ana models 
presented in the computer lessons supportive of the 
instructional methods used in the school in each of 
the following subjects? 

Reading 4.0 1 

Math 4.0 1 

Writing 4.0 1 

Science 2 

F.3. To what extent is there sufficient flexibility in the 
sequencing of the units that the computer program can 
be aligned with the school's classroom programs in 
each of the following subjects? 

Reading 3.0 1 

Math 3.0 1 

Writing 5.0 1 

Science 2 

F.4. To what extent do computer system lessons adequately 
prepare students for standardised achievement tests? 

Reading 4.0 1 

Math 4.0 1 

Writing ~~ 2 

Science ~~ 2 

F. 5. To what extent does the computer system curriculum 

meet your expectations? 

Reading 4.0 1 

Math 4.0 1 

Writing 4.0 1 

2 

G. l. To what extent is there increased student motivation 4.0 1 

and interest in your school as a result ot students 
participating in the computet learning system? 
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To what extent has the computer program led to highe 
student achievement 7 



Reading 
Math 
Writing 
Science 

To what extent has the computer system promoted 
student class participation? 

To what extent his ;.he computer system promoted 
greated parent interest and involvement in the 
school? 

To what extent has the computer system enhanced 
teacher effectiveness? 

To what extent has the computer system enhanced 
teacher communication and cooperation? 
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Administrator Survey 
Pittston School District (N=2) 



A. A. 
8.1. 

B. 2. 

B.3. 
B . A . 
B.5. 

B.6. 

B.7. 

B.8. 

B.9. 

B.10. 

B.ll. 
B.12. 

B.13. 

B.14. 

B.15. 

B.16. 

9 
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Item 

To what extent are you experienced in using 
computers? 

I don't know enough about the computer learning 
sy stem. 

I am concerned about the attitude of students toward 
the computer system. 

I know of other approaches that might work better. 

I have very limited knowledge about computers. 

I am concerned about the effects of the system on 
students . 

I am concerned about the changes this system will 
cause teachers to make in their classrooms. 

I am concerned about how much time and paperwork 
the computer system takes. 

I am concerned about how the scheduling of students 
on the system works out. 

I am concerned about the time and energy commitments 
the computer system requires. 

I am concerned about the knowing how to use the 
hardware . 

I am concerned about knowing how to use the software. 

I am concerned about the curriculum content of the 
computer learning system. 

I am concerned about the instructional approach used 
by the computer learning system. 

I am concerned about the training I receive on the 
computer learning system. 

I am concerned about the follow-up technical 
assistance I receive on the computer learning system. 

I am concerned about how well the computer system's 
curriculum aligns with our district curriculum and 
testing. 




B*17* I am concerned about the use of computer printouts 
from the system* 

B.18» I would like to know how this system is better than 
what we had before* 

B*19* I am not interested in learning more about the system* 

B.20. I am concerned about the value of instruction 
students receive on the system. 

D*l. How many hours of training did you receive to prepare 
you, personally, for your role in use of the computer 
system? 

D.2. To what extent do you feel the training adequately 

prepared you personally, for your roL in use of the 
computer system? 

D*4* To what extent do you feel the training adequately 
prepared your staff for use of the computer system? 

D*5, To what extent has the inservice training your staff 
received adequately covered the following areas? 

Reading 
Math 
Writing 
Science 

Higher Order Thinking Skills 
Use of Student results reports 

Integration of computer curriculum with classroom 
program 

Hands-on experience with the computer system 
Scheduling of students 

D*7* To what extent do you feel the follow-up technical 
assistance provided after training was adequate to 
meet the needs of you and your staff? 

D*8* To what extent has there been a need for follow-up 
technical assistance in the following areas7 

Reading 
Math 
Writing 
Science 

Higher Order Thinking Skills 
Use of Student results reports 

Integration of computer curriculum with classroom 
program 

Hands-on experience with the computer system 
Scheduling of students 



4*0 



4*0 
5*0 
1*0 
1.0 
1*0 
1.0 
2*0 

3*5 
3*5 

2*0 



4*0 

1*0 1 

1*0 1 

1*0 I 

1.0 1 

1*0 1 

1*0 1 

1.0 1 
1*0 
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D.9. 



To what extent have the tollowing documents met your 
staff's needs? 



Curriculum Guides 4.0 
Teacher Guides 4.0 
Teacher Handbooks 4.0 

E.2. To what extent are the reports clearly organized and 3.5 
easy to use? 

E.3. To what extent have the reports enabled your staff to: 

Diagnose student needs 3.5 

Individualize instruction 4.0 

Group students by ability 4.0 

Inform parents of student progress 4.0 

E.4. To what extent have the reports been useful to you 3.5 
as an administrator? 

E.5. To what extent has the computer system resulted in 
reallocation of time in the following areas? 

Reading 4.5 

Math 4.5 

Writing 1.0 1 

Science 1.0 1 

E.6. To what extent has the computer system resulted in 
increased time-on-task for all students in the 
following subject areas? 

Reading 4.5 
Math 4.5 
Writing 1.0 1 

Science 1.0 1 

E.7. To what extent do students enjoy working on their 5.0 
computer lessons? 

E.8. To what extent do you observe students working on 4.5 
lessons at the computer? 

E.9. To what extent has the computer system hardware been 4.5 
reliable and dependable? 

E.10. To what extent has implementation and operation of 1 . f 5 

the computer system created administrative problems 
for you? 
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F.l To what extent is the computer curriculum sufficiently 
comprehensive to provide adequately for the learning 
needs of students in each of the following subjects? 

Reading 2.5 

Math 4.5 

Writing 1.0 1 

Science 1.0 1 

F.2. To what extent are the learning strategies and models 
presented in the computer lessons supportive of the 
instructional methods used in the school in each of 
the following subjects? 

Reading 2.5 
Math A. 5 

Writing 1.0 1 

Science 1.0 1 

F.3. To what extent is there sufficient flexibility in the 
sequencing of the units that the computer program can 
be aligned with the school's classroom programs in 
each of the following subjects? 

Reading 2.5 
Math A. 5 

Writing 1.0 1 

Science 1.0 1 

F.4. To what extent do computer system lessons adequately 
prepare students for standardized achievement tests? 

Reading 2.5 

Math i* . 5 

Writing 1.0 1 

Science 1.0 1 

F. 5. To what extent does the computer system curriculum 

meet your expectations? 

Reading 2.5 

Math i*. 5 

Writing 1.0 1 

Science 1.0 1 

G. l. To what extent is there increased student motivation 4.5 

and interest in your school as a result of students 
participating in the computer learning system? 
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G.2. To what extent has the computer program led to higher 
student achievement? 



Reading 2.5 
Math 

Writing 1.0 1 

Science 1.0 1 

G.3. To what extent has the computer system promoted 4.0 
student class participation? 

G.4. To what extent has the computer system promoted 4.0 
greated parent interest and involvement in the 
school? 

G.5. To what extent has the computer system enhanced 4.0 
teacher effectiveness? 

G.6. To what extent has the computer system enhanced 4.0 
teacher communication and cooperation? 
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Teacher Survey 
Hatboro-Horsham Middle School (N=9) 



Item 

A. 4. To what, extent are you experienced in using 3.1 

computers? 

B. l. I don't know enough about the computer learning 2.9 

system. 

B.2. I am concerned about the attitude of students toward 2.8 
the computer system. 

B.3. I know of other approaches that might work better. 2.3 

B.4. I have very limited knowledge about computers. 2.1 

B.5. I am concerned about the effects of the system on 2.0 
students . 

B.6. I am concerned about the changes this system will 3.0 
cause teachers to make in their classrooms. 

B.7. I am concerned about how much time and paperwork 3.2 
the computer system takes. 

B.8. I am concerned about how the scheduling of students 3.7 
on the system works out. 

B.9. I am concerned about the time and energy commitments 3.3 
the computer system requires. 

B.10. I am concerned about the knowing how to use the 2.6 
hardware . 

B.ll. I am concerned about knowing how to use the software. 3.0 

B.12. I am concerned about the curriculum content of the 3.0 
computer learning system. 

B.13. I am concerned about the instructional approach used 2.8 
by the computer learning system. 

B.14. I am concerned about the training I receive on the 2.6 
computer learning system. 

B.15. I am concerned about the follow-up technical 2.8 
assistance I receive on the computer learning system. 

B.16. I am concerned about how well the compute: system's 3.3 
curriculum aligns with our district curriculum and 
testing. 



Mean NR 
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B.17. I am concerned about the use of computer printouts 2.7 
from the system. 

B«18> I would like to know how this system is better than 2.3 1 

what we had before. 

B.19. I air not interested in learning more about the system. 2.3 1 

B.20. i am ccncerned about the value of instruction 2.9 
student a receive on the system. 

D.2. To what e tent do you feel the training adequately 2.8 
prepared you personally, for your role in use of the 
computet* system? 

D. 3. To what extent has the inservice training you 

received adequately covered the following areas? 

Reading 2,6 U 

Math 3.8 2 

Writing 2,2 A 

Science 1»7 6 

Higher Order Thinking Skills 2,7 3 

Use of Student results reports 2.5 1 

Integration of computer curriculum with classroom 2.0 2 
program 

Hands-on experience with the computer system 3. A 1 

Scheduling of students 1.7 2 

D*4. To what extent have the following documents met 
your needs? 

Curriculum Guides 3.3 1 

Teacher Guides 3.3 1 

Teacher Handbooks 3.0 2 

E. 2. To what extent are the reports clearly organized and 1.8 1 

easy to use? 

E.3. To what extent have the reports enabled you to: 

Diagnose student needs 3.4 1 

Individualize instruction 3.1 2 

Group students by ability 2.9 2 

Inform parents of student progress 3.0 2 

E.4. To what extent do computer system student reports 

agree with your own evaluations of student classwork? 

Reading 6 

Math 3 

Writing 8 

Science 9 
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E.5, 



To what extent has the computer system resulted in 
reallocation of time in the following areas? 



Reading 8 

Math 3 

Writing 8 

Science 9 

E.6, To what extent has the computer system resulted in 
increased time-on-task for all students in the 
following subject areas7 

Reading 4.0 7 

Math 3 .4 2 

Writing 4.0 8 

Science 9 

E,7, To what extent do students enjoy working on their 2,0 
computer lessons? 

E.8. To what extent do you observe students working on 3.3 1 

lessons at the computer? 

E. 9. To what extent are you comfortable with the use of 2,6 

the computer system? 

F. l To what extent is the computer curriculum sufficiently 

comprehensive to provide adequately for the learning 
needs of students in each of the following subjects? 

Reading 3,7 6 

Math 3,7 2 

Writing 4.0 8 

Science 9 

F.2. To what extent are the learning strategies and models 
presented in the computer lessons supportive of the 
instructional methods used in the school in each of 
the following subjects? 

Reading 3.3 6 

Math 3.3 2 

Writing 4*0 8 

Science 9 

F.3* To what extent is there sufficient flexibility in the 
sequencing of the units that the computer program can 
be aligned with the school's classroom programs in 
each of the following subjects? 

Reading 2.5 7 

Math 3.0 5 

Writing 4.0 8 

Science 9 
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F.4. 



To what extent do computer system lessons adequately 
prepare students for standardized achievement tests? 



Reading 3.0 8 

Math 3.3 3 

Writing 9 

Science — 9 

F, 5. To what extent does the computer system curriculum 

meet your expectations? 

Reading 4.5 7 

Math 4.2 4 

Writing 5.0 8 

Science 9 

G. l. To what extent is there increased student motivation 4.0 1 

and interest in your school as a result of students 
participating in the computer learning system? 



Reading 
Math 
Writing 
Science 

G.2. To what extent has the computer program led to higher 
student achievement? 



Reading 3.0 8 

Math 2.7 2 

Writing 4.0 8 

Science ~~ 9 

G.3. To what extent has the computer system served to 
enhance your own classroom teaching? 

Reading 3.5 7 

Math 3.0 3 

Writing 4.0 8 

Science 9 



3.5 7 
3.3 2 
4.0 8 
9 
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Teacher Survey 
Pittston Area Middle School (N=6) 



A. 4 . 

B. l. 

B.2. 

B.3. 
B.4. 
B.5. 

B.6. 

B.7. 

B.8. 

B.9. 

B.10. 

B.H. 
B.12. 

B.13. 

B.14. 

B.15. 

B.16. 

ERIC 



Item 

To what extent are you experienced in using 
computers? 

I don't know enough about the computer learning 
system. 

I am concerned about the attitude of students toward 
the computer system. 

I know of other approaches that might work better. 

I have very limited knowledge about computers. 

I am concerned about the effects of the system on 
students . 

I am concerned about the changes this system will 
cause teachers to make in their classrooms. 

I am concerned about how much time and paperwork 
the computer system takes. 

I am concerned about how the scheduling of students 
on the system works out, 

I am concerned about the time and energy commitments 
the computer system requires. 

I am concerned about the knowing how to use the 
hardware . 

I am concerned about knowing how to use the software. 

I am concerned about the curriculum content of the 
computer learning system. 

I am concerned about the instructional approach used 
by the computer learning system. 

I am concerned about the training I receive on the 
computer learning system. 

I am concerned about the follow-up technical 
assistance I receive on the computer learning system. 

I am concerned about how well the computer system's 
curriculum aligns with our district curriculum and 
testing. 




B,17, I am concerned about the use of computer printouts 2.3 
from the system. 

B.18. I would like to know how this system is better than 1.3 
what we had before. 

B.19. I am not interested in learning more about the system. 1.7 

B.20. I am concerned about the value of instruction 2.5 
students receive on the system. 

(Teachers did not respond to the remaining sections of the survey.) 
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Parent Survey 
Hatboro-Horsham Middle School (N=46) 



Item Mean NR 

A. 5. How well informed are you about the new computer 2.3 
learning system at your child* s school? 

A. 6. Have you been invited by the school to visit your 1.6 
child's classroom? 

A. 7. Did you visit your child's classroom? 1.7 

A. 8. Have you observed your child working at the 2.0 
computer in school? 

A. 9. Have you attended a parent conference at school? 1.3 

A. 10. Did the teacher ever discuss your child's progress 1.9 
on the computer program? 

A. 11. Does your child talk about work on the computer 1.3 
after coming home from school? 

A. 12. Has work on the computer made your child more 1.5 
enthusiastic about school? 

A, 13. Does your child like school botter this year 1.5 
because of work on the computer? 

A. 14. Is there a computer in your home? 1.6 

B. l. My child feels good about the academic progress 3.6 

being made on the computer. 

B.2, My child's teacher keeps rne informed about my 1.6 
child's progress on the computer. 

B.3. Parents are encouraged to visit the school. 2.8 

B.4. My child frequently says positive things ajout 3.3 
the computer program. 

B.5. My child looks forward to days that are scheduled 3.6 
for work on the computer. 

B.6. My child's attitude toward school has improved 2 . c ) 

because of the computer program. 

B.7. My child's attitude toward learning has improved 3.0 
because of the computer program. 

B.8. My child's reading skills have improve because of 2.9 
the computer program. 

O 38 



B.9. My child's math skills have improved because of 3,4 
the computer program* 

B.10. My child's writing skills have improved because of 2.7 
the computer program. 
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Computer Home Use Survey 
Hatboro-Horsham Middle School (N=5) 



Item Mean NR 

B.2. How difficult was it for your child to learn how to 5.0 1 

use the computer system? 

B.3. How often did your child use the computer system 3.5 
at home? 

B.7. How frequently did you work with your child during 1,6 
use cf the computer system? 

B.8. To what extent did your child enjoy working on the 4.0 
computer system? 

B.9. To what extent did your child learn from using the 4-0 
computer system at home? 

B.10. To what extent did your child's math skills improve 3.8 1 

from working on the computer system at home? 

B.ll. To what extent did your child's reading skills 3.0 2 

improve from working on the computer system at home? 

B.12. To what extent did your child's writing skills 2.7 2 

improve from working on the computer system at home? 

B.13. To what extent did your child's understanding of 1.0 1 

science improve from working on the computer system 
at home? 
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Computer Home Use Survey 
Pittston Area Middle School (N=4) 



Item Mean NR 

B.2. How difficult was it for your child to learn how 5.0 
to use the computer system? 

B.3. How often did your child us the computer system 4.8 
at home? 



B.7. How -requently did you work with your child during 2.0 
use of the computer system? 

B.8. To what extent did your child enjoy working on the 4.8 
computer system? 

B.9. To what extent did your child learn from using the 5.0 

computer system at home? 

B.10. To what extent did your child's math skills improve 4.3 
from working on the computer system at home? 

B.ll. To what extent did your Ud's reading skills improve 4.3 
from working on the computer system at home? 

B.12. To what extent did your child's writing skills improve 2.3 1 
from working on the computer system at home? 

B.13. To what extent did your child's understanding of 3.0 1 

science improve from working on the computer system 
at home 7 



9 

ERLC 



41 

38 



Student Survey 
Hatboro-Horsham Middle School (N=68) 



Percent Responding 

Item Yes No NR 

1. Do you like school? 60 37 3 

2. Is the computer easy to u?e? 99 1 

3. Is working on the computer fun? 84 13 3 

4. Do computers make it fun to learn? 85 13 2 

5. Do you learn a lot on the computer? 74 2? 1 

6. Do computers make school subjects more 57 41 2 
interesting? 

7. Do you get bored working on the computer 53 46 1 
by yourself? 

8. Do you need much help when working on the 4 96 
computer? 

9. Does your computer give you help when you 81 18 1 
need it? 

10. Does the computer help you correct your mistakes? 90 10 

11. Do you have to hurry when you work on the 12 88 
computer? 

12. Do you like computer work better than written 93 7 
assignments? 

13. Is it important to do well on your computer 85 12 3 
assignments? 

14. Does working on the computer help you do better 65 29 6 
in school? 

15. Does your teacher know whether you make mistakes 54 40 6 
on your computer assignments? 

16. Do you get good grades when you work hard in 88 9 3 
school? 

17. Do you like going to the computer? 85 15 

18. Would you like to go to the computer more often? 66 34 

19. Have you worked on a computer in school before 78 22 
this year? 



620. 


Do you have a computer at home? 


47 


53 




21. 


Do you like using the computer at school? 


87 


13 




22. 


Has the computer helped you to learn math better? 


75 


25 




23. 


Has the iputer helped you to read better? 


32 


SO 


1Z 


24. 


Has the computer helped you to write better? 


13 


72 


15 


25. 


Has the computer helped you to understand 
science better? 


3 


71 


26 


26. 


Can you do most of the computer lessons without 
help from anyone else? 


93 


7 




27. 


Are the computer lessons interesting? 


69 


28 


3 


28. 


Do your computer lessons help you do work in 
the classroom better? 


57 


A3 




29. 


Is your time with the computer the best part of 
your day? 


29 


69 


2 


30. 


Do your parents think you are learning from the 


53 


32 


15 



computer? 
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Student Survey 
Pittston Area Middle School (N=150) 



Percent Responding 

Item Yes No NR 

1. Do you like school? 51 45 4 

2. Is the computer easy to use? 95 5 

3. Is working on the computer fun? 94 6 

4. Do computers make it fun to learn? 91 9 

5. Do you learn a lot on the computer? 77 22 1 

6. Do computers make school subjects more 80 19 1 
interesting? 

7. Do you get bored working on the computer by 26 73 1 
yourself? 

8. Do you need much help when working on the 8 91 1 
computer? 

9. Does your computer give you help when you 83 16 1 
need it? 

10 . Does the computer help you correct your mistakes? 81 16 3 

11. Do you have to hurry when you work on the 23 73 4 
computer? 

12. Do you like computer work better than written 91 7 2 
assignments? 

13. Is it important to do well on your computer 75 19 6 
assignments? 

14. Does working on the computer help you do better 61 34 5 
in school? 

15. Does your teacher know whether you make mistakes 53 41 5 
on your computer assignments? 



16. Do you get good grades when you work hard in 
school? 



81 15 



17. Do you like going to the computer? 89 7 4 

18. Would you like to go to the computer more often? 89 9 2 

19. Have you worked on a computer in school before 93 5 2 
tliis year? 
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20. Do you have a computer at home? 33 64 3 

21. Do you like using the computer at school? 89 8 3 

22. Has the computer helped you to learn math better? 65 29 6 

23. Has the computer helped you to read better? 37 55 8 

24. Has the computer helped you to write better? 17 77 6 

25. Has the computer helped you to understand 25 68 7 
science better? 

26. Can you do most of the computer lessons without 85 11 4 
help from anyone else? 

27. Are the computer lessons interesting? 80 15 5 

28 . Do your computer lessons help you do w;>rk in 61 35 4 
the classroom better? 

29. Is your time with the computer the best part 41 57 2 
of your day? 

30. Do your parents think you are learning from the 63 27 10 
computer? 
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